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Executive Summary 

The major challenge that school 
districts face is to improve the learning 
of all students, not just in individual 
schools, but across the entire system of 
schools. Duval County, Florida has 
embarked on a remarkable journey to 
implement standards-based reform in 
schools throughout the district with the 
intent to systematically improve 
teaching and learning across its 150 
schools. In this study, we examine the 
impact of Duval's efforts by 
documenting the county's changes in 
state test performance in comparison to 
other, similar Florida counties. Through 
these analyses, we explore the 
hypothesis that Duval's strategies for 
increasing the quality of teaching and 
learning across its system of schools is 
paying off in terms of faster gains in 
student learning relative to other 
counties within the state. 

In this study, we examine 
elementary and middle school reading, 
writing, and mathematics results from 
the spring of 1999 to the spring of 2002 
on the Florida Comprehensive 
Assessment Test (FCAT) in Duval 
relative to seven other counties in 
Florida. We purposefully chose to begin 
our study with the results of the 1998- 
1999 school year because that year 
represented the year before Duval 
County began implementing standards- 
based reform. The 2002 data are the 
latest state test data available. The 
comparison counties include Broward, 
Flillsborough, Lee, Orange, Palm Beach, 
Pinellas, and Polk, which were the most 
reasonably comparable counties in 
Florida. From each of these counties, we 



included all school-level data that were 
available from the Florida Department 
of Education. 

This is a study of the effects of a 
school system's efforts to bring about 
widespread improvements in student 
learning. While these school-level data 
do not allow us to examine growth in 
the performance of individual students 
over multiple years, we can compare 
growth in school-level scores for 
different cohorts of students. Thus, for 
example, we examine school-level 
changes in fourth-grade performance 
from one year to the next in Duval 
County relative to other Florida 
counties, and then examine these 
changes over successive cohorts of 
fourth graders. With this design, we 
argue that we are comparing the 
changes in student performance that are 
produced by the school systems within 
which students and schools reside. For 
example, if fourth-grade scores in one 
county increase more than fourth-grade 
scores in other counties during a given 
year, and this pattern persists over 
multiple years, we believe that it is 
reasonable to argue that the school 
system is producing these sustained 
higher levels of performance. 

Overall, during the four years of the 
study, average state test performance in 
Duval generally grew from year to year, 
as it did in most other counties in the 
state. In comparison to other counties, 
the patterns that emerged from this 
analysis are strong and persistent gains 
in Duval's elementary school 
performance relative to its peers, and 
virtually no differences in its middle 
school test performance relative to 
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other, similar Florida counties. In some 
cases, the elementary school patterns 
represented Duval's pulling ahead of its 
counterparts, while in other cases, it 
represented Duval's catching up with 
the comparison counties. More 
specifically: 

. In elementary school writing, 

Duval's overall pattern was one of 
moving from performing worse than 
virtually all its peer counties to 
performing as well as or better than 
them. From 1999 to 2000 in fourth- 
grade writing, Duval performed 
significantly worse than five counties 
and no different from two. By 2002, 
Duval performed significantly better 
than one county and no different 
from the other six. 

. In elementary school reading, 
Duval's overall pattern was one of 
moving from similar to superior 
performance relative to the 
comparison counties. In fourth-grade 
reading in 1999, Duval performed 
significantly better than two 
counties, no different from four, and 
worse than one. By 2002, Duval 
performed significantly better than 
all seven comparison counties. 

. In elementary school mathematics, 
Duval also showed greater 
improvements in performance 
compared to its peers. In 2000, the 
district performed significantly 
better than one county, significantly 
worse than another, and no different 
from five. By 2002, the district was 
performing significantly better than 
five counties and no different from 
two. 



. Although Duval middle school state 
test performance generally improved 
over the time of this study, middle 
school reading and mathematics 
results showed no differences in 
improvement relative to the other 
Florida counties we examined. 
Duval's middle school reading and 
mathematics performance was 
virtually indistinguishable from that 
of other comparable counties from 

2000 to 2002. In eighth-grade 
writing, Duval performed worse 
over time relative to other counties 
in the state. From 1999 to 2000, 

Duval was performing no different 
from six of its peer counties and 
significantly worse than one. From 

2001 to 2002, Duval performed 
significantly worse than four of the 
comparison counties and no 
different from three. 

These results indicate impressive 
positive effects in elementary schools, 
while there are indistinguishable 
differences or negative effects in middle 
schools. These analyses are worth 
extending as students who were in 
elementary schools during the period of 
this study move into middle schools. 

Overall, these results provide 
evidence that Duval's efforts to 
systematically change the practices of 
teachers and school leaders across its 
system are improving the achievement 
of its elementary schools at a faster rate 
than in other comparable districts. In 
middle schools, to date, growth in 
performance has been comparable to 
that of other counties. 
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Introduction 

The scaling-up of educational reform 
is the holy grail of educational 
improvement. While there are 
numerous examples of individual 
teachers and schools that dramatically 
change their instructional and 
organizational practices and produce 
remarkable gains in student learning, 
producing similar changes in many 
classrooms and schools is the ultimate 
goal of education reform. The individual 
cases usually reflect changes in the ways 
that teachers and students engage with 
and construct knowledge and the role of 
school structures in facilitating this 
process. Yet organizationally, educators 
have been unable to produce these 
changes on a large scale. This, according 
to Elmore (1996), is the crux of the 
problem of scale, which can be seen as 
"the need to change the core of 
schooling in ways that result in more 
students receiving engaging instruction 
in challenging academic content" (p. 6). 
In this study, we describe the efforts of 
Duval County, Florida to improve the 
content and instruction that all its 
children receive, with a focus on the 
impact of these efforts on student 
learning over time. 

After this introduction, we briefly 
describe Duval's strategies to change the 
content and environment of teaching 
and learning in the district. Then, we 
explain our method for selecting a 
sample of similar Florida counties 
against which to compare Duval's 
performance over time. Next, we 
portray additional demographics of 
Duval and the comparison counties, and 
then present the methods for our 



analysis, including a description of the 
data sources and our analytical 
approach. We conclude this report as we 
first present and then discuss the results 
of our analyses in reading, writing, and 
mathematics in both elementary and 
middle schools. 

The Duval Story 

Duval County is the 14* largest 
school district in the United States, 
serving approximately 127,000 students 
in 150 schools. In June 1998, John Fryer, 
a retired Air Force Major General, was 
appointed superintendent. Early in his 
tenure. Fryer selected America's Choice, 
a Comprehensive School Reform 
program offered by the National Center 
on Education and the Economy (NCEE), 
as a means of systematically improving 
teaching and learning. Fryer described 
his initial challenge as a way to quickly 
infuse expertise into the system, "I did 
not have the time to build a reform from 
the ground up and thought that NCEE 
represented my best opportunity to 
acquire the capacity fast" (personal 
interview, October 2001). 

In the 1999-2000 and 2000-2001 
school years, 61 Duval elementary and 
middle schools adopted the America's 
Choice design, representing about 40% 
of the county's schools and creating 
what the superintendent called a 
"critical mass." As Fryer described, he 
never seriously considered purchasing 
America's Choice for all schools in the 
county. At any rate, the county could 
not have afforded to buy the model for 
each of its schools. And even if it could 
have afforded to do so, it still faced the 
problem of how to support schools once 
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they completed the three-year America's 
Choice program. During the 1998-1999 
school year, Duval also began 
implementing a new mathematics 
curriculum. Connected Math, in sixth and 
seventh grades. All eighth-grade 
teachers were trained to use Connected 
Math in 1999-2000. In 1999-2000, Duval 
began a county-wide adoption of 
Investigations in third through fifth 
grades. In 2000-2001, all K-2 teachers 
were trained to use Investigations. 

At the end of the 2002-2003 school 
year, the last Duval schools that 
adopted the design in 2000-2001 will be 
completing their final year of America's 
Choice. There are no plans to continue 
purchasing the design. During the last 
several years, Duval has begun to 
explicitly develop its internal capacity to 
support all of the county's schools to 
implement its version of standards- 
based reform, which is strongly 
influenced by the philosophy and 
strategies of NCEE, infused with 
Duval's own beliefs and approaches, 
and shaped by the requirements and 
accountability system of the state of 
Elorida. The county is in a fascinating 
transitional period in which it seeks to 
increasingly refine its reforms, articulate 
them to stakeholders, and develop the 
internal capacity to deepen the work. 

Reform efforts in Duval's high 
schools are only in their nascent stages. 
Although Duval has implemented some 
changes within its high schools, 
involving some of them as pilot sites for 
the America's Choice high school design 
for example, the county has only 
recently begun to attend seriously to the 
distinct challenges of improving 



teaching and learning in high schools. 
Eor this reason, we have not included 
high schools in our analyses since we 
would not expect them to be performing 
differently than their counterparts in 
other counties at this point. In 
subsequent years, as reform takes 
deeper root, it would be reasonable to 
expect differential changes in Duval 
high schools as well in comparison to 
high schools in comparable Elorida 
counties. 

Duval's strategy includes the 
following elements: 

• A focus on standards for student 
performance. Standards represent a 
clear and commonly understood set 
of expectations for what children 
should know and be able to do at 
each grade level, and a focal point 
for teachers to engage in discussions 
of student work. 

• A retooling of the curriculum in 
English /language arts, mathematics, 
and science that provides more 
opportunities for students to learn 
the material they are expected to 
master. In English /language arts, the 
county has made a strong 
commitment to standards-based 
literacy, including hour-long blocks 
of authentic reading and writing 
instruction and practice 
supplemented by skills instruction. 

In mathematics, the county has 
adopted more challenging curricula, 
including Investigations for 
elementary schools and Connected 
Math for middle schools. In science, 
the county is honing its curriculum 
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to reduce repetition and increase 
cognitive demand over time. 

. An intensive and unprecedented 
commitment to the professional 
development of teachers, school 
leaders, and county administrators. 
This includes ongoing literacy 
training at each grade level in each 
school, the development of teacher 
leaders in mathematics and science, 
full-time standards coaches in each 
school, leadership training for 
principals, and ongoing workshops 
for county leaders. 

. The fostering of adult learning 
communities where teachers and 
school and county leaders engage in 
their own ongoing development as 
learners. 

• A special emphasis on raising the 
performance of the county's lowest- 
performing students and schools. 
Informed by systematic and regular 
examination of data, schools have 
developed a variety of intervention 
and assistance strategies to identify, 
serve, and monitor students who are 
not meeting standards. 

• A county-wide monitoring system to 
track the implementation of 
standards that further articulates 
expectations for school progress and 
provides feedback to both school 
and county leaders about their 
progress toward the county's goals. 

Together, these elements and others 
form the foundation of a comprehensive 
strategy that is ever undergoing 
revision, reformulation, and refinement. 



However, this strategy would not be 
nearly as compelling if there were not 
evidence that the county's efforts were 
producing systematic gains in student 
learning. It is this evidence that we seek 
with this study. 

Sample 

There are a total of 67 counties/ 
public school districts in the state of 
Florida. More than two-thirds of these 
are rural counties serving relatively 
small numbers of students. To select 
counties that were comparable to the 
more urban Duval County, we 
examined county-level data on the 14 
largest counties in Florida (including 
Duval) — those serving more than 
20,000 students. These data are available 
from the Florida State Department of 
Education in its school indicators report, 
but they are reported separately for 
elementary, middle, and high schools. 
We chose to use each county's 1998-1999 
elementary school demographic data as 
a proxy for each system. We analyzed a 
number of elementary school 
characteristics including: number of 
students, percentage of students 
receiving lunch assistance, fourth-grade 
reading performance, and school 
operating costs. After examining the 
demographic data, we removed Miami- 
Dade from consideration because it is 
the proverbial "800 pound gorilla" of 
Florida counties, having 60% more 
elementary school students than the 
next largest county and two-and-a-half 
times as many students as Duval. From 
the remaining 12 counties, we selected 7 
for inclusion in the study because they 
were the most similar to Duval in terms 
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Table 1. 1998-1999 Elementary School Characteristics Used 
in the Selection of Comparison Counties 


County 


State Rank 
by Size 


Number of 
Elementary 
Students 


Percent of Students 
Receiving 

Free/Reduced-price 

Lunch 


Percent of Fourth 
Graders Scoring 
at or Above 
Standard in 
Reading 


Operation 

Costs 


Broward 


2 


114,667 


44% 


49% 


$4,608 


Hillsborough 


3 


79,391 


57% 


53% 


$4,666 


Palm Beach 


4 


71,263 


43% 


51% 


$5,028 


Orange 


5 


68,741 


53% 


47% 


$4,337 


Duval 


6 


65,402 


56% 


54% 


$4,417 


Pinellas 


7 


52,274 


47% 


63% 


$4,637 


Polk 


8 


37,968 


62% 


49% 


$4,614 


Lee 


12 


27,114 


53% 


59% 


$4,928 



of the four characteristics examined. 
These were Broward, Hillsborough, Lee, 
Orange, Palm Beach, Pinellas, and Polk 
counties. The other five counties 
(Brevard, Volusia, Seminole, Pasco, and 
Escambia) were considerably different 
on county demographics than Duval, 
particularly in terms of county size and 
the percent of students eligible for 
free /reduced-price lunch assistance. 
Table 1 shows the descriptive statistics 
for each of the eight counties that were 
maintained in the study. 

County Demographics 

The final sample of counties 
represents a rich geographic and ethnic 
cross-section of Florida. The counties in 
our sample range from Broward in the 
Fort Lauderdale area of southeastern 
Florida, the second largest public school 
system in the state; to Orange County, 
the fifth largest county in the state. 



covering the Orlando area; to 
Hillsborough, serving the Tampa area; 
to smaller Lee County in the 
southwestern part of the state around 
Fort Myers. 

Demographic details about the 
counties are provided in Tables 2 and 3, 
decomposed for elementary and middle 
schools (the subjects of our analyses) in 
the eight counties in the 1998-1999 
school year. In these tables, we also 
report average test scores from the 1998- 
1999 school year, spring 1999 
administration for fourth, fifth, and 
eighth graders in the eight counties. 

In 1998-1999, the year before 
standards-based reform was introduced 
into Duval County schools, elementary 
schools in Duval, Broward, 
Hillsborough, Lee, Orange, Palm Beach, 
Pinellas, and Polk counties had similar 
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Table 2. Mean Characteristics and Standard Deviations for Eiementary Schoois in Eight County 
Schooi Districts in Fiorida in the 1998-1999 Schooi Year, Prior to Standards-based Reform 





Broward 


Duval 


Hills- 

borough 


Lee 


Orange 


Palm 

Beach 


Pinellas 


Polk 


No. of 
Schools 


130 


103 


111 


37 


99 


81 


81 


62 


Absent 
21+ Days 


7.58 

(2.58) 


9.37 

(3.52) 


7.84 

(3.57) 


6.29 

(3.14) 


7.97 

(3.25) 


7.63 

(3.91) 


6.75 

(2.98) 


9.40 

(4.27) 


Class Size 


26.34 

(2.59) 


23.06 

(3.89) 


25.28 

(17.32) 


22.68 

(1.96) 


20.91 

(3.16) 


25.43 

(2.75) 


23.06 

(1.29) 


22.61 

(2.07) 


Free/Red. 

Lunch 


46.37 

(28.48) 


60.73 

(21.60) 


58.61 

(24.29) 


53.07 

(16.86) 


53.99 

(21.68) 


44.35 

(30.07) 


46.07 

(19.41) 


61.81 

(17.28) 


LEP 


9.42 

(6.69) 


1.00 

(4.11) 


11.15 

(11.71) 


8.80 

(6.36) 


8.61 

(8.46) 


13.59 

(10.48) 


1.73 

(3.27) 


4.43 

(4.75) 


Special 

Education 


12.30 

(3.99) 


17.03 

(5.38) 


13.77 

(5.90) 


17.04 

(4.85) 


17.29 

(10.07) 


15.16 

(10.80) 


18.18 

(5.12) 


13.53 

(6.11) 


Mobility 


30.54 

(14.59) 


45.22 

(15.54) 


52.13 

(90.59) 


34.16 

(13.02) 


48.77 

(30.93) 


28.20 

(11.19) 


33.42 

(15.07) 


39.71 

(15.89) 


School 

Graded 


2.80 

(0.88) 


2.72 

(0.90) 


2.99 

(0.87) 


3.24 

(1.01) 


2.79 

(0.97) 


2.86 

(1.12) 


3.38 

(0.94) 


2.82 

(0.91) 


School Size 


871.83 

(230.07) 


633.40 

(263.03) 


713.03 

(251.15) 


732.81 

(228.12) 


693.91 

(212.45) 


851.90 

(254.94) 


645.36 

(172.86) 


589.92 

(178.02) 


Minority 


54.96 

(28.29) 


50.30 

(26.47) 


48.30 

(23.75) 


32.40 

(14.14) 


50.39 

(25.98) 


50.18 

(28.20) 


26.31 

(12.08) 


36.69 

(14.04) 


4th-grade 

Reading*' 


281.74 

(22.97) 


289.23 

(20.35) 


285.23 

(24.82) 


291.86 

(19.99) 


276.86 

(26.41) 


281.62 

(26.86) 


292.49 

(31.31) 


283.97 

(20.11) 


5th-grade 

Math*> 


304.02 

(21.20) 


298.40 

(18.54) 


306.11 

(20.56) 


307.81 

(16.07) 


297.18 

(22.89) 


303.06 

(22.64) 


304.34 

(25.02) 


298.40 

(18.95) 


4th-grade 

Writing'' 


2.96 

(0.28) 


2.88 

(0.31) 


3.10 

(0.33) 


2.91 

(0.33) 


2.81 

(0.35) 


2.88 

(0.32) 


3.10 

(0.53) 


2.98 

(0.41) 



Notes: 

3 School grades were converted to a numerical scale where A=5, B=4, C=3, D=2, and F=l. FCAT Sunshine State 
Standards Test. ■= FCAT Florida Writing Assessment. 



percentages of students with severe 
absenteeism (i.e., absent 21 or more 
days), average class sizes, average 
school accountability grades, and 
average fourth-grade writing 
performance. The eight counties also 
had comparable percentages of students 
involved in special education programs, 
ranging from 12% in Broward to 18% in 



Pinellas. However, Palm Beach (14%), 
Hillsborough (11%), and Broward, Lee, 
and Orange (each 9%) counties had an 
appreciably higher percentage of 
Limited English Proficient (LEP) 
students than in Duval County (1%) and 
Pinellas County (2%). 



6 



The Impact of Standards-based Reform in Duval County, Florida: 1999-2002 



The percentage of students eligible 
for free or reduced-price lunch ranged 
from a low of 44% in Palm Beach 
County to a high of 62% in Polk County. 
Duval County had a higher percentage 
of students eligible for free or reduced- 
price lunch assistance than Broward, 
Palm Beach, and Pinellas counties (61% 
compared to about 45%) and Orange 
and Lee counties (61% compared to 
about 53%), and a similar percentage to 
Polk (62%) and Hillsborough (59%) 
counties. Additionally, elementary 
student mobility was the highest in 
Hillsborough County (52%), 
considerably higher than Palm Beach, 
Broward, Pinellas, Lee, and Polk 
counties (28% to 40%) and somewhat 
higher than Duval County (45%). 
Broward and Palm Beach counties had 
the largest average school size, with 872 
and 852 students, respectively, and Polk 
County had the smallest average school 
size with 590 students, which was 
similar to Duval County (633 students). 

The percentage of minority student 
enrollment ranged from 26% in Pinellas 
County to 55% in Broward County, with 
similar percentages for Duval, 
Hillsborough, Orange, and Palm Beach 
counties (about 50%). For 1998-1999 
average reading achievement, Lee and 
Pinellas County fourth graders 
performed the highest (292), but similar 
to Duval (289), and Orange County 
fourth graders performed the lowest 
(277); Broward, Hillsborough, Palm 
Beach, and Polk counties all performed 
equivalently, scoring around 283. In 
mathematics. Orange County fifth 



graders also performed the lowest on 
average (297), but similar to Duval and 
Polk County fifth graders (298); the 
other five counties had similar average 
mathematics scores, ranging from 303 
(Palm Beach) to 308 (Lee). Overall, on 
average, Duval County elementary 
schools were most similar to Polk 
County elementary schools on all of the 
characteristics examined, except for 
percentage of minority students. 

For middle schools in 1998-1999, as 
shown in Table 3, the eight counties 
varied on average characteristics, except 
for average school accountability grade. 
The percentage of students absent 21 or 
more days ranged from 8% in Polk 
County to 18% in Duval County, with 
the other counties between 11% and 
16%. Average class size in language arts 
and mathematics both ranged from 21 
students per classroom in Hillsborough 
County to 29 students per classroom in 
Broward County, with an average of 24 
students for language arts and 26 
students for mathematics; Duval County 
averaged 26 students per classroom for 
both language arts and mathematics. 
Duval County had the highest 
percentage of students eligible for free 
or reduced-price lunch assistance (53%), 
higher than Palm Beach, Pinellas, and 
Broward counties, but similar to 
Hillsborough, Orange, Polk, and Lee 
counties. In addition, Duval County had 
the highest average middle school 
student mobility rate at 50%, noticeably 
higher than all of the other counties, and 
Palm Beach had the lowest mobility rate 
at 24%. 
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Table 3. Mean Characteristics and Standard Deviations for Middle Schools in Eight County School 
Districts in Florida in the 1998-1999 School Year, Prior to Standards-based Reform 





Broward 


Duval 


Hills- 

borough 


Lee 


Orange 


Palm 

Beach 


Pinellas 


Polk 


No. of 
Schools 


34 


27 


36 


17 


31 


26 


21 


22 


Absent 
21+ Days 


16.15 

(5.68) 


18.49 

(11.79) 


14.76 

(6.60) 


11.05 

(4.90) 


15.64 

(15.00) 


11.20 

(5.42) 


13.21 

(5.67) 


8.50 

(8.98) 


Class Size 
LA^ 


29.11 

(3.66) 


25.61 

(4.40) 


21.39 

(2.46) 


20.72 

(3.25) 


22.97 

(4.29) 


26.22 

(3.68) 


26.13 

(1.38) 


22.68 

(3.41) 


Class Size 
Math 


29.42 

(4.22) 


25.81 

(3.55) 


20.83 

(3.06) 


22.90 

(1.63) 


25.23 

(7.69) 


27.77 

(3.25) 


26.73 

(1.00) 


26.94 

(14.11) 


Free/Red. 

Lunch 


41.44 

(21.63) 


52.81 

(21.30) 


50.36 

(20.35) 


46.34 

(18.94) 


50.35 

(20.92) 


35.85 

(27.57) 


37.01 

(16.97) 


48.42 

(15.59) 


LEP 


5.89 

(3.79) 


1.25 

(2.43) 


6.36 

(5.02) 


5.51 

(2.61) 


6.84 

(5.34) 


7.12 

(5.93) 


1.52 

(2.77) 


2.18 

(2.98) 


Special 

Education 


10.27 

(2.36) 


17.80 

(6.62) 


14.24 

(4.24) 


17.35 

(3.19) 


19.56 

(15.70) 


16.01 

(4.31) 


18.06 

(5.63) 


13.63 

(5.28) 


Mobility 


27.83 

(9.82) 


50.05 

(24.47) 


38.00 

(14.80) 


38.68 

(15.12) 


40.43 

(37.02) 


23.57 

(9.57) 


29.07 

(12.97) 


39.24 

(17.95) 


School 

Grade» 


3.09 

(1.19) 


2.77 

(0.95) 


3.03 

(0.66) 


3.41 

(1.18) 


2.93 

(0.91) 


3.24 

(0.97) 


3.67 

(0.80) 


3.18 

(1.01) 


School Size 


1562.00 

(348.13) 


1090.78 

(407.53) 


979.47 

(388.98) 


901.35 

(242.76) 


1042.52 

(438.38) 


1288.46 

(378.93) 


1147.38 

(248.81) 


778.14 

(350.02) 


Minority 


57.61 

(23.64) 


55.13 

(24.07) 


51.99 

(20.30) 


33.53 

(16.98) 


53.44 

(21.93) 


56.80 

(27.07) 


29.49 

(14.21) 


37.74 

(17.29) 


8th-grade 

Reading'’ 


280.22 

(33.66) 


284.37 

(25.07) 


291.44 

(24.34) 


292.64 

(21.83) 


281.52 

(30.75) 


292.03 

(28.00) 


282.87 

(38.84) 


281.85 

(29.14) 


8th-grade 

Math" 


281.41 

(35.44) 


277.87 

(31.89) 


294.30 

(30.90) 


293.07 

(32.29) 


283.52 

(33.07) 


289.97 

(31.20) 


281.50 

(45.52) 


282.48 

(31.85) 


8th-grade 

Writing"' 


3.01 

(0.55) 


3.24 

(0.38) 


3.46 

(0.42) 


3.10 

(0.37) 


3.07 

(0.44) 


2.94 

(0.65) 


3.05 

(0.67) 


3.09 

(0.61) 



Notes: = LA=language arts. '’School grades were converted to a numerical scale where A=5, B=4, C=3, D=2, and F=l. 
FCAT Sunshine State Standards Test, FCAT Florida Writing Assessment. 



The percentage of students enrolled 
in special education programs ranged 
from 10% in Broward County to 20% in 
Orange County, with the other counties 
between 14% and 18%. Duval County, 
compared to Broward, Hillsborough, 
Lee, Orange, and Palm Beach counties. 



had a lower percentage of LEP students 
(1% compared to 6% to 7%), but a 
similar percentage to Pinellas and Polk 
counties (2%). Broward had the largest 
average school size (about 1,560 
students) and Polk County had the 
smallest average school size (about 780 
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students), with the other counties 
ranging from approximately 900 
students (Lee) to about 1,560 students 
(Broward). Broward County also had 
the highest percentage of minority 
students (58%), but comparable to Palm 
Beach, Duval, Orange, and Hillsborough 
counties, and Pinellas had the lowest 
percentage (29%). 

For 1998-1999 eighth-grade average 
reading achievement, Lee County 
performed the highest (293), but 
equivalent to Palm Beach and 
Hillsborough counties, and Broward 
performed the lowest (280), but 
equivalent to Orange, Polk, Pinellas, and 
Duval counties. Eighth-grade 
mathematics results were similar to 
reading, as Hillsborough, Lee, and Palm 
Beach performed similarly high on 
average (294, 293, and 290, respectively), 
and Duval, Broward, Pinellas, Polk, and 
Orange counties performed similarly 
lower (278, 281, 281, 282, and 284, 
respectively). In writing, most eighth 
graders performed the same on average 
(around 3.1), except for Hillsborough 
County which performed the highest, 
scoring approximately 3.5. Overall, on 
average, Duval County middle schools 
had a high percentage of students 
eligible for free or reduced-price lunch, 
high student mobility, high minority 
student enrollment, low LEP student 
enrollment, and low reading and 
mathematics performance compared to 
the other counties. Duval County 
middle schools seemed most similar to 
Orange County middle schools on 
average demographics and student 
performance. 



Across the eight counties, academic 
performance in the six subjects/ grades 
examined in this study show a general 
trend of improvement. That is, while 
there is some bouncing around of year- 
to-year test score averages, the raw 
county average scale scores are 
generally improving over time. This 
could reflect the results of programs 
within counties, the state's methods for 
calculating results, changes in the 
composition of county schools, or some 
combination of these factors. Duval's 
test results also show trends that are 
generally positive. As an example, we 
show fourth- and eighth-grade average 
county reading scores in Table 4. 

The only adjustment made to these 
numbers was weighting them by the 
number of students tested in each 
school. As the reader can see from the 
table, Duval's performance was flat 
(fourth grade) or declining (eighth 
grade) from spring 1999 to spring 2000, 
and then grew over the following two 
years. These results are fairly typical. 
That is, overall, Duval showed gains in 
average performance from year to year 
relative to itself during the period 
examined in this study. This indicates 
that Duval's unadjusted performance in 
the three subjects we examined — 
reading, writing, and mathematics — 
has continued to grow over the time 
that standards-based reform was being 
implemented within the county. 

Because there were some differences in 
the direction of county changes in 
performance from year to year, we resist 
talking about gains in this report, but 
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Table 4. Changes in Average Fourth- and Eighth-grade FCAT Reading Scaie Scores from 



Spring 1999 to Spring 2002 for Duvai and the Seven Comparison Counties 





1999 


Fourth Grade 
2000 2001 


2002 


1999 


Eighth Grade 
2000 2001 


2002 


Broward 


285 


292 


301 


304 


295 


294 


297 


300 


Duval 


294 


294 


298 


303 


293 


284 


293 


293 


Hillsborough 


289 


297 


303 


301 


297 


294 


302 


301 


Lee 


292 


298 


301 


304 


302 


297 


297 


299 


Orange 


280 


282 


287 


289 


292 


283 


289 


288 


Palm Beach 


285 


287 


297 


300 


301 


292 


296 


294 


Pinellas 


297 


298 


301 


299 


302 


291 


303 


304 


Polk 


285 


290 


296 


292 


289 


286 


290 


285 



rather use a more cautious approach of 
talking about changes in performance. 
The full set of raw county average test 
scores for fourth and eighth grade are 
shown in Appendix A. 

These results still do not take into 
account the very real demographic 
differences between these counties, 
which is why we have adopted a more 
sophisticated analytical strategy using 
regression modeling. This strategy is 
described in the following section. 

Methods 

In this section, we describe the 
sources for the data that we used to 
produce these analyses and our 
analytical approach. 

Data Sources 

All of the school-level data used in 
the analyses for this study were 
downloaded from the Florida State 
Department of Education's website. The 



FCAT Sunshine State Standards 
reading, writing, and mathematics test 
data for grades 4, 5, and 8 were found at 
www.firn.edu/ doe /sas /feat/ fcinfopg.h 
tm. The demographic data used as 
covariates in the analytic models were 
taken from the Florida School Indicators 
Report: 1998-1999 and were downloaded 
from info.doe.state.fl.us/fsir/, except 
for percent minority which was 
calculated from the 1998-1999 Common 
Core of Data collected by the National 
Center for Education Statistics 
( nces.ed.gov / ced /) . District-level 
profiles for the latest school year, 2001- 
2002, were also available online at 
www.firn.edu / doe / eias / Amove / count 
y.htm . 

FCAT scores for students in fourth, 
fifth, and eighth grades were collected 
for the 1998-1999, 1999-2000, 2000-2001, 
and 2001-2002 school years. This 
standardized test was administered in 
the spring of each school year. Both 
norm-referenced and standards- 
referenced tests comprise the FCAT 
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system. For this study, we only used the 
standards-referenced scores on the 
FC AT because they are more aligned 
with the standards-based reform being 
evaluated. The FCAT Sunshine State 
Standards scale scores for reading and 
mathematics range from 100 to a 
possible 500 and the writing scores 
range from 1 to 6. All of the schools 
within each county that had available 
1998-1999 demographic and test data 
were included in the analyses. Schools 
with missing demographic data were 
not included. 

Statistical Analyses 

In order to assess the impact of 
Duval's standards-based reform efforts 
on student achievement, standardized 
test scores in reading, writing, and 
mathematics for fourth-, fifth-, and 
eighth-grade cohorts from the eight 
counties were analyzed over four 
consecutive years: 1998-1999, 1999-2000, 
2000-2001, and 2001-2002. Broward, 
Hillsborough, Lee, Orange, Palm Beach, 
Pinellas, and Polk counties served as 
comparison counties for Duval. All of 
the data used in the analyses were 
school averages as individual-level 
student performance data were not 
publicly available. 

Ordinary least squares regression 
models were used to compare average 
school-level achievement in Duval 
County to average school-level 
achievement in the seven comparison 
counties in reading, writing, and 
mathematics after adjusting for school 
demographic characteristics and prior 
achievement. Models were estimated 
separately for each grade level and 



subject area combination using PROC 
REG in SAS 8.2. The basic model 
equation follows: 

Y = + pi(PRIOR ACHIEVEMENTjj + 

p^CSEVERE ABSENTEEISM), + 
p3(CLASS SIZE), + p,(LUNCH 
ASSISTANCE), + PsCLEP), + 
p, (SPECIAL EDUCATION), + 
p,(MOBILITY), + p,(SCHOOL 
GRADE), + p, (SCHOOL SIZE), + 
p,„(PERCENT MINORITY),, + 
p„(BROWARD)„ + 
p,, (HILLSBOROUGH),, + 

P,3(LEE),3 + p„(ORANGE)„ + 
p„(PALM BEACH),, + 
p„(PINELLAS)„ + P„(POLK)„ + r 

In the model, the outcome Y is the 
school's average spring test score for a 
particular grade level and subject area, 
and r is the random error term. The 10 
school-level covariates included in the 
model are: average prior spring test 
score, percentage of students who were 
absent 21 or more days, average class 
size, percentage of students eligible for 
free or reduced-price lunch, percentage 
of students who were classified as 
Limited English Proficient, percentage 
of students in special education 
programs, percentage of students 
moving into or out of a school during 
the 1998-1999 school year, school grade 
(A-F) assigned under the state 
accountability system,^ number of 
students enrolled in a school in fall 1998 
(i.e., school size), and the combined 
percentage of students who were 
African American, Hispanic, or Native 
American. County-level means and 



' School grades were converted to a numerical 
scale where A=5, B=4, C=3, D=2, and F=l. 
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standard deviations for these variables 
were provided in Tables 2 and 3 above. 

Dummy indicators for schools in 
Broward, Hillsborough, Lee, Orange, 
Palm Beach, Pinellas, and Polk counties 
were also included in each model. The 
reference group, the omitted group to 
which all other counties are compared, 
was Duval County schools. This 
analytical approach is similar to a 
hierarchical linear modeling (HLM) 
approach (Raudenbush & Bryk, 2002); 
however, there is a fixed effect for 
district instead of a random effect as in a 
traditional HLM. This enables us to 
make explicit comparisons of the seven 
districts to Duval County. 

Our analyses include three separate 
statistical tests for each grade level and 
subject area tested. Because these 
samples are not independent across 
tests, this increases the chance of 
making a Type I error; that is, 
incorrectly finding a significant 
difference when in fact no difference 
exists. Statisticians commonly use 
adjustments to counteract this potential 
bias (Miller, 1981). We used a Bonferroni 
correction to adjust the significance 
levels (i.e., p-values) of the estimates 
downward in order to account for the 
number of statistical tests performed on 
each independent sample under study. 
This does not change the effect sizes, but 
provides a more stringent test of 
significance so that Type I error is not 
inflated. These conservative adjustments 
are reflected in the results tables and in 
the full models reported in Appendix B 
of this report. 



Thus, our analytical approach was to 
use regression analyses, repeated across 
years, correcting for the multiple tests. 
We considered looking solely at gain 
scores from spring 1999 to spring 2002, 
and estimating a linear growth model 
for all years between 1999 and 2002, but 
we felt that these approaches could 
mask trends within this time period 
because these models embed the 
assumption that change from one year 
to the next is constant. For example, if 
there were differences between 1999 and 
2002, these could be due to consistent 
improvements each year, initial 
improvements that flattened out in 
subsequent years, or changes that did 
not appear until the last year that was 
examined, and we wanted to identify 
these differences. Therefore, we chose to 
report results for individual years, 
correcting for the additional number of 
statistical tests performed, in order to 
make apparent the trends within the 
period from 1999 to 2002. 

Results 

In this section, we report the relative 
changes in student performance for 
students in Duval County schools in 
comparison to the changes of students 
in other Florida counties after 
controlling for school characteristics and 
prior achievement. Findings are 
reported on the three-year impact of the 
standards-based reform in Duval 
County on elementary and middle 
schools in reading, writing, and 
mathematics. Results are reported 
separately by grade level, and by subject 
area. Also, the p-values reported have 
been conservatively corrected for 
multiple statistical tests as described 
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previously in this report. These 
coefficients are compared to the 
reference group of Duval County 
schools. Appendix B contains complete 
tables of regression coefficients and 
standard errors for each of the variables 
included in each of the models. All of 
the coefficients are in standard deviation 
units and represent the standardized 
effect size of the difference between 
Duval and each other county after 
statistically controlling for the 10 school 
baseline characteristics. Cohen (1977) 
considers effect sizes of .2 as small, .5 as 
moderate, and .8 as large. 

Elementary School 
Performance 

From 1998-1999, the year before 
Duval began its implementation of 
standards-based reform, to 2001-2002, 
Duval showed dramatic gains in 
performance in elementary school 
reading, writing, and mathematics test 
performance in comparison to other 
Florida counties. In this section, we 
detail the results in fourth-grade 
reading and writing, and fifth-grade 
mathematics performance from the 
1998-1999 school year to the 2001-2002 
school year. 

Fourth-grade Reading Performance 

In fourth-grade reading, the overall 
pattern shows that, at the begiiming of 
its implementation of standards-based 
reform in the 1998-1999 school year, 
Duval was performing roughly 
similarly to the comparison districts. 
However, by the 2001-2002 school year, 
Duval was outperforming the seven 
comparison districts. 



The patterns of gains in fourth-grade 
reading performance are shown 
graphically in Figure 1. The first set of 
bars (shown as 1999-2000) show test 
score changes in Duval compared to 
those of the seven other comparison 
counties during the first full year of 
standards-based reform in Duval 
County. The second set of bars (shown 
as 2000-2001) show test score changes in 
Duval compared to those of the seven 
other counties during the second year of 
standards-based reform in Duval. The 
third set of bars (shown as 2001-2002) 



Figure 1. Duval County’s Fourth-grade 
Reading Performance Compared to Seven 
Other Florida Counties 




show test score changes in Duval 
relative to those of the seven other 
counties during the third year of reform 
in Duval. Those bars above the zero axis 
represent counties significantly 
outperformed by Duval in that given 
year. Those bars below the zero axis 
represent counties which significantly 
outperformed Duval in that given year. 
All differences are represented as 
standardized units, and only those gains 
that are statistically significant are 
shown as differences. Thus, non- 
significant differences, whether they 
favor Duval or another county, are 
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represented as zero. Again, in this figure 
and in all subsequent figures, the reader 
can consider differences around .2 as 
small, .5 as moderate, and .8 as large. 

The first set of bars in Figure 1 
represent changes in Duval's fourth- 
grade FCAT reading performance in 
comparison to changes in the 
performance of fourth graders in the 
seven comparison counties from the end 
of the 1998-1999 school year to the same 
test given at the end of the 1999-2000 
school year. This year represents a 
pre/ post measure of the begiiming of 
Duval's implementation of standards- 
based reform as 1999-2000 was the first 
year of Duval's first cohort of America's 
Choice schools. Each bar in the graph 
represents the change in Duval's 
performance in that year relative to the 
changes of that comparison district. For 
example, as can be seen in the first set of 
bars in the graph, Duval changed no 
differently than Broward, Lee, Pinellas, 
or Polk counties. Hillsborough, by 
contrast, statistically outperformed 
Duval County by .14 of a standard 
deviation. Duval outperformed Orange 
and Palm Beach during its first year 
implementing standards-based reform. 

After Duval's second year of 
standards-based reform, represented by 
the middle section of Figure 1, Duval 
was begiiming to see even larger 
improvements in test performance in 
comparison to the other Florida 
counties. From the spring 2000 FCAT 
reading test to the spring 2001 FCAT 
reading test, fourth graders in Duval 
performed significantly better than did 
students in three comparison counties 
(Lee, Orange, and Pinellas), while 



performing no differently than four 
others (Broward, Hillsborough, Palm 
Beach, and Polk). Hillsborough, which 
had outgained Duval the previous year, 
was performing no differently than 
Duval after the second year of the 
implementation of standards-based 
reform in Duval. 

By 2002, after Duval's third year 
engaged in standards-based reform, the 
county was significantly outperforming 
all of the comparison districts. These 
differences are shown in the final set of 
bars in Figure 1. The magnitude of these 
differences in average school test 
performance ranged from small (.18 to 
.34) to moderate (.43 to .45). These effect 
sizes translate into a 4 to 11 point 
difference on the fourth-grade FCAT 
reading test in 2002. Thus, after 
controlling for school demographic 
characteristics and prior achievement, 
the average school performance in 
Duval was 299, Broward was 295, 
Orange was 291, and Polk was 288. 

Fourth-grade Writing Performance 

In contrast to the story in elementary 
school reading where Duval went from 
similar performance to outperforming 
all the comparison counties, the story of 
writing shows Duval catching up with 
other Florida counties. In fourth-grade 
writing, Duval initially performed 
significantly worse than comparison 
counties and ended up performing 
similarly. 
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The patterns in fourth-grade FCAT 
writing performance are shown in 
Figure 2. Changes in Duval's 
performance from the 1998-1999 school 
year, the year prior to the 
implementation of standards-based 
reform, to the 1999-2000 school year, are 
shown in the leftmost set of bars. Duval 
performed significantly worse than five 
of the comparison districts 
(Hillsborough, Lee, Palm Beach, 

Pinellas, and Polk), and performed no 
differently than two of the comparison 
counties (Broward and Orange). Thus, 
in the first year of standards-based 
reform, after controlling for school 
demographic characteristics and prior 
achievement, Duval was performing .09 
to .26 points worse than the comparison 
counties on the fourth-grade FCAT 
writing test. The average adjusted 
school performance was 3.03 in Duval, 
3.27 in Hillsborough, 3.29 in Pinellas, 
and 3.25 in Polk. 

Figure 2. Duval County’s Fourth-grade Writing 
Performance Compared to Seven Other 
Florida Counties 




99-00 00-01 01-02 



□ Broward ^Hillsborough alee □ Orange nPalm n Pinellas a Polk 



A year later, in 2001, Duval schools 
had caught up with, and in two cases 
surpassed, the schools in the 
comparison counties. In fact, Duval 
fourth graders were performing no 
different from students in five of the 
comparable counties (Broward, 
Hillsborough, Lee, Palm Beach, and 
Polk) and performing significantly 
better than students in Orange and 
Pinellas. These positive performance 
differences were generally maintained 
in 2002 when Duval fourth-grade 
students generally changed similarly in 
writing relative to the comparison 
counties. Duval continued to 
outperform Orange County, by about a 
quarter of a standard deviation, in 2002. 

Fifth-grade Mathematics 
Performance 

Duval's elementary school 
mathematics performance seemed 
largely indistinguishable from that of 
the comparison counties in the first two 
years of standards-based reform. From 
2001 to 2002, however, Duval schools 
outperformed most of the comparison 
counties' schools. 

Year-to-year changes in fifth-grade 
FCAT mathematics performance for 
Duval relative to the seven comparison 
counties are shown in Figure 3. There do 
not appear to be any notable patterns of 
systematic differences in relative county 
average test performance from 1999 to 
2000 or from 2000 to 2001, the first two 
years of standards-based reform in 
Duval County. Significant patterns of 
difference in performance start to 
appear from 2001 to 2002 as the changes 
in performance in Duval County were 
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Figure 3. Duval County’s Fifth-grade 
Mathematics Performance Compared to 
Seven Other Fiorida Counties 




significantly more positive than those of 
the other counties. 

From the 1998-1999 school year, the 
year before Duval began to implement 
standards-based reform, to the 1999- 
2000 school year, Duval's first year of 
implementation of standards-based 
reform, the county's mathematics 
performance was essentially no different 
from that of other counties in the state. 

In that year, Duval performed 
significantly better than Pinellas, 
significantly worse than Hillsborough, 
and no different from Broward, Lee, 
Orange, Palm Beach, and Polk counties. 

This non-descript pattern continued 
in 2001. From the spring 2000 FCAT 
administration to the spring 2001 
administration, Duval performed 
significantly better than Pinellas 
County, significantly worse than 
Broward and Hillsborough counties, 
and no different from Lee, Orange, Palm 
Beach, and Polk counties. 



In 2002, Duval really began to 
perform significantly better than its 
counterparts in fifth-grade mathematics. 
During the 2001-2002 school year, Duval 
fifth-grade mathematics performance 
was stronger than that of just about all 
of the comparison counties. Duval's 
fifth-grade performance was 
significantly stronger than that of five 
counties (Lee, Orange, Palm Beach, 
Pinellas, and Polk) and no different 
from two (Broward and Hillsborough). 
The magnitude of the differences was 
between two-tenths and three-tenths of 
a standard deviation, a small but 
consistent effect. This translates into a 
range of five to eight points, about seven 
points on average, on the fifth-grade 
FCAT mathematics test. The average 
school performance for Duval was 310 
after controlling for school demographic 
characteristics and prior achievement. 

Middle School Performance 

Duval middle school FCAT test 
performance from 1998-1999 to 2001- 
2002 was largely flat relative to that of 
the comparison counties in reading and 
mathematics. There was, however, some 
evidence of degradation of performance 
in eighth-grade writing. In this section, 
we examine the results in eighth-grade 
reading, writing, and mathematics 
performance from the 1998-1999 school 
year, the year before standards-based 
reform was implemented in Duval 
County, to 2001-2002. 
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Eighth-grade Reading Performance 

Over the timeframe of this study, 
there were no patterns of systematic 
differences in the performance of Duval 
eighth graders in reading in comparison 
to the school-average performance of 
other counties. As shown in Figure 4, 
there is only one case in which Duval's 
eighth-grade performance was 
significantly different from that of 
eighth-grade performance in other 
comparison counties. From the spring 
2000 administration of the eighth-grade 
FCAT reading test to the spring 2001 
administration, Duval performed 
significantly worse than Hillsborough 
by .19 of a standard deviation. In all 
other years, there were no differences in 
changes in Duval's performance relative 
to the changes in performance of the 
other seven counties. 

Figure 4. Duval County’s Eighth-grade 
Reading Performance Compared to Seven 
Other Fiorida Counties 
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Eighth-grade Writing Performance 

Eighth-grade writing is the one 
subject and grade level where there was 
a pattern that suggests that Duval is 



falling behind the performance of other 
Florida counties. The results in eighth- 
grade FCAT writing for Duval relative 
to the comparison counties are shown in 
Figure 5. 

Figure 5. Duvai County’s Eighth-grade Writing 
Performance Compared to Seven Other 
Fiorida Counties 
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From the spring of 1999, the year 
before Duval began implementing 
standards-based reform, to the spring of 
2000, after the first year of reform, 

Duval was essentially performing no 
differently than the other seven counties 
in Florida. The exception to this was 
Hillsborough, which outperformed 
Duval by .44 of a standard deviation, a 
moderate effect. 

This pattern of no differences 
between Duval and the other seven 
Florida counties persisted in 2001, as can 
be seen in the middle section of Figure 
5. Only Hillsborough County continued 
to perform significantly different from 
Duval, gaining .39 of a standard 
deviation more than Duval. In 2002, 
Duval's performance dropped off 
relative to the other Florida counties. 
From the spring 2001 administration of 
the eighth-grade FCAT writing test to 
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the spring 2002 administration, Duval's 
performance was significantly worse 
than that of four of the comparison 
counties (Broward, Hillsborough, 
Pinellas, and Polk) while no different 
from three (Lee, Orange, and Palm 
Beach) after controlling for differences 
in the demographic characteristics and 
prior achievement of the schools in each 
of the counties. This translates into a .14 
to .25 point lower test performance on 
the eighth-grade FCAT writing test, 
which had a minimum possible score of 
1 and a maximum possible score of 6. 
The average adjusted school 
performance was 3.58 in Duval, 3.83 in 
Broward, 4.05 in Hillsborough, 3.95 in 
Pinellas, and 3.84 in Polk. The trend in 
writing performance bears monitoring 
to see if the 2002 performance is an 
anomaly or part of a trend. 



By 2002, the third year of standards- 
based reform, Duval was performing no 
differently than all seven of the 
comparison counties. 

Figure 6. Duval County’s Eighth-grade 
Mathematics Performance Compared to 
Seven Other Fiorida Counties 




Discussion 



Eighth-grade Mathematics 
Performance 

Eighth-grade mathematics 
performance, like eighth-grade reading 
performance, shows no real patterns of 
difference between Duval County and 
the seven comparison counties. The 
differences from 2000 through 2002 are 
shown in Figure 6. In 2000, after 
controlling for performance in 1999, 
Duval performed significantly better 
than one county (Pinellas), significantly 
worse than one county (Hillsborough) 
and no different from the five other 
counties. 

In 2001, after two years of standards- 
based reform, Duval was performing 
similarly to six of the comparison 
counties, and continued to perform 
significantly worse than Hillsborough. 



In this report, we have set out to 
quantify the impact of the standards- 
based reform efforts of Duval County on 
student learning by examining the 
trends in school achievement scores in 
Duval compared to seven other Florida 
counties that were similar in terms of 
size, per-pupil spending, poverty, and 
achievement prior to the implement- 
ation of Duval's reforms. We have 
examined trends in school-level 
achievement over time from the 1998- 
1999 school year through the 2001-2002 
school year. We chose the 1998-1999 
school year as a baseline because it 
represents the year before the 
implementation of the first cohort of 
America's Choice in Duval, which was 
the beginning of Duval's concerted 
attempts to change the educational 
practices which influence student 
learning. At the time of these analyses. 
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the 2001-2002 data represent the latest 
data available from the state of Florida. 
Our analytic strategy involved 
comparing the changes in performance 
in Duval to that of other counties from 
one year to the next at each of the grade 
levels tested by the state (4, 5, and 8), 
and repeating this analysis across 
multiple cohorts at each grade level. 
Thus, for example, we compared school- 
level changes in fourth-grade reading 
performance, controlling for school 
demographic characteristics and prior 
achievement, across multiple years. We 
contend that if students in Duval are 
performing better than students in 
other, similar counties in a grade level 
over multiple years, then these effects 
are produced by the school system 
within which they attend. 

The results of this analysis show that 
changes in Duval's elementary school 
performance in reading, writing, and 
mathematics were substantially stronger 
than the changes in performance in the 
seven comparison counties from 1999 to 
2002. Although the patterns differed 
somewhat by subject and grade level, 
Duval elementary schools either caught 
up to or surpassed the aimual 
differences in performance of schools in 
the comparison counties, even after 
controlling for both demographic 
differences and prior performance. 

These patterns suggest that something 
systemic is occurring in Duval 
elementary schools that is resulting in 
persistent outperformance relative to 
other counties. We believe that these 
systematic differences can be reasonably 
attributed to Duval's standards-based 
reform efforts. 



In contrast to elementary schools, 
there were no trends of differential 
performance in Duval County middle 
schools in reading and mathematics 
relative to the performance of the 
comparison counties. In writing, there 
was a relative decline in performance 
from 2001 to 2002. The raw performance 
data from these years (see Appendix A) 
indicate that Duval was generally 
making gains in middle school 
performance during these years, but 
that these gains were similar to those of 
the comparison counties. Thus, while 
the middle schools have held their own, 
they did not outperform middle schools 
in the comparison counties. 

Depending on how one looks at it, 
the lack of middle school differences can 
be interpreted as Duval holding its own 
amidst change or indicate the need for 
Duval to re-think its middle school 
strategy. Duval's standards-based 
reform initiative has had more difficulty 
taking root in its middle schools than it 
has in its elementary schools. In 
mathematics, for example, Duval 
initially had difficulties implementing 
Connected Math in its middle schools, 
where many teachers found it difficult 
to teach with this more conceptual 
curriculum. So it is possible that the 
county's continued improvement — on 
par with the comparison counties — 
even in the face of a challenging 
transition, represents a major 
achievement. It is also possible that it is 
harder to change the learning habits of 
middle school students, who are more 
acculturated to traditional forms of 
teaching. If these possibilities hold true, 
then the effects in Duval middle schools 
may become visible as middle school 
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teachers become more accustomed to 
the new instructional approaches of the 
county and elementary students begin 
to enter the middle schools. If either of 
these possibilities are reasonable, then 
the county should faithfully persist with 
its efforts. 

It is also possible that middle schools 
represent distinctly different 
environments from elementary schools 
and the organization and culture of 
middle schools may be more 
intransigent to the county's standards- 
based reform. If this is the case, then 
Duval County will need to tweak its 
strategies to change the instructional 
and organizational habits of its middle 
schools. 

Despite the uneven progress across 
grade levels, the patterns of effect 
documented in this study provide 
substantial evidence that the county's 
standards-based reform efforts have 
produced significant changes in the 
learning experiences of students. Even 
as its transformation has just begun, 
Duval County's efforts to date have had 
positive and measurable impacts on the 
learning of the county's students. 
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Appendix A 



Table A-1. Mean Scale Scores in Fourth-grade Reading, Weighted for the Number of Students 
Tested, for each of the Eight Counties from Spring 1999 to Spring 2002 



County 


1999 


2000 


2001 


2002 


Broward 


285 


292 


301 


304 


Duval 


294 


294 


298 


303 


Hillsborough 


289 


297 


303 


301 


Lee 


292 


298 


301 


304 


Orange 


280 


282 


287 


289 


Palm Beach 


285 


287 


297 


300 


Pinellas 


297 


298 


301 


299 


Polk 


285 


290 


296 


292 



Table A-2. Mean Scale Scores in Fourth-grade Writing, Weighted for the Number of Students 
Tested, for each of the Eight Counties from Spring 1999 to Spring 2002 



County 


1999 


2000 


2001 


2002 


Broward 


3.0 


3.2 


3.5 


3.5 


Duval 


2.9 


3.0 


3.4 


3.4 


Hillsborough 


3.1 


3.4 


3.6 


3.6 


Lee 


2.9 


3.2 


3.4 


3.5 


Orange 


2.9 


3.1 


3.2 


3.2 


Palm Beach 


2.9 


3.2 


3.4 


3.5 


Pinellas 


3.2 


3.4 


3.5 


3.5 


Polk 


3.0 


3.3 


3.5 


3.4 



22 The Impact of Standards-based Reform in Duval County, Florida: 1999-2002 



Table A-3. Mean Scale Scores in Fifth-grade Mathematics, Weighted for the Number of Students 
Tested, for each of the Eight Counties from Spring 1999 to Spring 2002 



County 


1999 


2000 


2001 


2002 


Broward 


306 


315 


323 


327 


Duval 


302 


311 


307 


315 


Hillsborough 


308 


321 


321 


323 


Lee 


308 


320 


317 


319 


Orange 


300 


309 


303 


307 


Palm Beach 


306 


318 


318 


321 


Pinellas 


307 


314 


309 


317 


Polk 


301 


312 


311 


313 


Table A-4. 


Mean Scale Scores in Eighth-grade Reading, Weighted for the Number of Students 
Tested, for each of the Eight Counties from Spring 1999 to Spring 2002 


County 


1999 


2000 


2001 


2002 


Broward 


295 


294 


297 


300 


Duval 


293 


284 


293 


293 


Hillsborough 


297 


294 


302 


301 


Lee 


302 


297 


297 


299 


Orange 


292 


283 


289 


288 


Palm Beach 


301 


292 


296 


294 


Pinellas 


302 


291 


303 


304 


Polk 


289 


286 


290 


285 
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Table A-5. Mean Scale Scores In Eighth-grade Writing, Weighted for the Number of Students 
Tested, for each of the Eight Counties from Spring 1999 to Spring 2002 



County 


1999 


2000 


2001 


2002 


Broward 


3.3 


3.6 


3.6 


3.9 


Duval 


3.3 


3.6 


3.7 


3.7 


Hillsborough 


3.6 


4.0 


4.1 


4.2 


Lee 


3.2 


3.8 


3.7 


3.8 


Orange 


3.2 


3.6 


3.5 


3.7 


Palm Beach 


3.3 


3.6 


3.6 


3.7 


Pinellas 


3.4 


3.8 


3.9 


4.0 


Polk 


3.3 


3.7 


3.8 


3.9 


Table A-6. Mean Scale Scores in Eighth-grade Mathematics, Weighted for the Number of Students 
Tested, for each of the Eight Counties from Spring 1999 to Spring 2002 


County 


1999 


2000 


2001 


2002 


Broward 


296 


306 


313 


310 


Duval 


290 


297 


302 


301 


Hillsborough 


303 


314 


318 


317 


Lee 


306 


313 


309 


309 


Orange 


295 


300 


304 


301 


Palm Beach 


301 


307 


311 


307 


Pinellas 


305 


303 


313 


311 


Polk 


291 


299 


301 


297 
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Appendix B 



Table B-1. Regression Coefficients from the Full Models for Reading Achievement by Year and 

Grade Level (with corrected significance leveis) 



Variable 


Spring 1999 to Spring 2000 
Grade 4 Grade 8 


Spring 2000 to Spring 2001 
Grade 4 Grade 8 


Spring 2001 to Spring 2002 
Grade 4 Grade 8 


Intercept 


.06 


-.16*** 


-.01 


- 20*** 


.13** 


-.15*** 




(.04) 


(.04) 


(.04) 


(.04) 


(.04) 


(.04) 


Pre-test Score 


.50*** 


.65*** 


.55*** 


.61*** 


gy*** 


.75*** 




(.04) 


(.06) 


(.03) 


(.06) 


(.04) 


(.06) 


Absent 21 


-.06* 


-.04 


-.07** 


.02 


-.06- 


-.03 




(.02) 


(.03) 


(.02) 


(.03) 


(.03) 


(.03) 


Class Size 


06** 


-.04 


-.04- 


.00 


-.05* 


.01 




(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


Free/Red. Lunch 


-.18*** 


.03 


- 12** 


-.09 


-.11* 


-.08 




(.04) 


(.05) 


(.04) 


(.05) 


(.04) 


(.04) 


LEP 


-.01 


-.03 


.01 


-.05* 


.00 


-.00 




(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


Special Education 


-.07** 


-.24*** 


-.03 


-.13* 


-.06* 


-.01 




(.02) 


(.05) 


(.02) 


(.05) 


(.03) 


(.05) 


Mobility 


-.03 


-.05 


-.11* 


-.05 


- 17** 


-.01 




(.05) 


(.04) 


(.04) 


(.04) 


(.05) 


(.04) 


School Grade 


.04 


.01 


j_j_*** 


.02 




.02 




(.03) 


(.03) 


(.02) 


(.03) 


(.03) 


(.02) 


School Size 


-.02 


-.00 


-.00 


-.01 


-.03 


-.04 




(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


Minority 


-.13*** 


-.17*** 


-.08- 


-11** 


-.04 


.03 




(.03) 


(.04) 


(.03) 


(.04) 


(.04) 


(.04) 


Hillsborough 


.14- 


.10 


-.00 


.19* 


-27*** 


.07 




(.06) 


(.07) 


(.06) 


(.06) 


(.07) 


(.06) 


Orange 


-21** 


-.05 


-26*** 


.06 


-.34*** 


-.10 




(.07) 


(.07) 


(.06) 


(.07) 


(.07) 


(.06) 


Broward 


-.09 


.15 


.04 


-.11 


-.18* 


.06 




(.07) 


(.07) 


(.06) 


(.07) 


(.07) 


(.06) 


Lee 


.02 


.04 


-.19- 


-.06 


-.21- 


.05 




(.09) 


(.08) 


(.08) 


(.08) 


(.09) 


(.08) 


Palm Beach 


-.25** 


.04 


-.06 


-.04 


- 32*** 


-.07 




(.08) 


(.07) 


(.07) 


(.07) 


(.08) 


(.07) 


Polk 


-.03 


-.01 


.02 


-.09 


-.45*** 


-.12 




(.07) 


(.08) 


(.07) 


(.08) 


(.08) 


(.07) 


Pinellas 


-.12 


-.13 


-25*** 


.11 


-.43*** 


.11 




(.07) 


(.07) 


(.06) 


(.07) 


(.07) 


(.07) 


R2 


.81 


.92 


.82 


.93 


.78 


.92 



- p<.10, * p < .05, ** p < .01, *** p < .001 after Bonferroni correction 
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Table B-2. Regression Coefficients from the Full Models for Writing Achievement by Year and 
Grade Level (with corrected significance leveis) 

Spring 1999 to Spring 2000 Spring 2000 to Spring 2001 Spring 2001 to Spring 2002 
Variabie Grade 4 Grade 8 Grade 4 Grade 8 Grade 4 Grade 8 



Intercept 


-.46*** 


.05 


-.19** 


-.12 


- 19** 


-.20*** 




(.06) 


(.05) 


(.06) 


(.07) 


(.06) 


(.06) 


Pre-test Score 


.55*** 


62* * * 


.55*** 


.61*** 


54*** 


.58*** 




(.04) 


(.07) 


(.03) 


(.08) 


(.03) 


(.06) 


Absent 21 


.04 


-.04 


-.07- 


-.00 


.02 


.00 




(.03) 


(.03) 


(.03) 


(.04) 


(.03) 


(.04) 


Class Size 


-.01 


.02 


-.04 


.04 


-.00 


-.04 




(.02) 


(.03) 


(.02) 


(.03) 


(.02) 


(.03) 


Free/Red. Lunch 


_ 23* * * 


.02 


-.09 


-.02 


-.09 


-.10 




(.06) 


(.06) 


(.06) 


(.07) 


(.06) 


(.06) 


LEP 


.02 


-.07* 


-.00 


-.05 


.02 


-.05 




(.03) 


(.02) 


(.03) 


(.03) 


(.03) 


(.03) 


Special Education 


-.04 


_ 23* * * 


-.06 


-.08 


-.06 


-.09 




(.03) 


(.06) 


(.03) 


(.07) 


(.03) 


(.06) 


Mobility 


.01 


-.05 


.00 


.07 


- 18** 


-.09 




(.07) 


(.05) 


(.06) 


(.06) 


(.06) 


(.05) 


School Grade 


.02 


.05 


.06 


.08 


.18*** 


.02 




(.04) 


(.03) 


(.03) 


(.04) 


(.03) 


(.03) 


School Size 


-.01 


-.04 


.02 


-.08- 


.00 


-.04 




(.03) 


(.03) 


(.03) 


(.04) 


(.03) 


(.03) 


Minority 


.15** 


-.03 


-.08 


-.07 


.06 


.06 




(.05) 


(.04) 


(.05) 


(.05) 


(.05) 


(.05) 


Hillsborough 


67*** 


44*** 


-.08 


.39** 


.12 


47*** 




(.09) 


(.08) 


(.09) 


(.11) 


(.09) 


(.09) 


Orange 


.12 


.10 


-.45*** 


-.09 


-.23* 


.09 




(.09) 


(.09) 


(.09) 


(.10) 


(.09) 


(.09) 


Broward 


.11 


-.14 


-.03 


-.03 


-.01 


.25* 




(.09) 


(.09) 


(.09) 


(.11) 


(.09) 


(.09) 


Lee 


4j_** 


.20 


-.07 


-.01 


-.02 


.05 




(.12) 


(.11) 


(.12) 


(.13) 


(.12) 


(.11) 


Palm Beach 


.25- 


-.08 


-.11 


.05 


-.04 


.11 




(.11) 


(.09) 


(.10) 


(.11) 


(.10) 


(.10) 


Polk 


.61*** 


.04 


.04 


.19 


-.09 


.26* 




(.10) 


(.09) 


(.10) 


(.11) 


(.10) 


(.10) 


Pinellas 


72*** 


.16 


-.38*** 


.20 


.02 


37*** 




(.10) 


(.09) 


(.10) 


(.11) 


(.09) 


(.10) 


R2 


.57 


.87 


.59 


.82 


.61 


.85 



- p<.10, * p < .05, ** p < .01, *** p < .001 after Bonferroni correction 
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Table B-3. Regression Coefficients from the Fuii Models for Mathematics Achievement by Year and 

Grade Level (with corrected significance leveis) 

Spring 1999 to Spring 2000 Spring 2000 to Spring 2001 Spring 2001 to Spring 2002 
Variabie Grade 5 Grade 8 Grade 5 Grade 8 Grade 5 Grade 8 



Intercept 


_ 20* * * 


-.21*** 


-.30*** 


_ 41*** 


_ 17*** 


-.20*** 




(.04) 


(.04) 


(.04) 


(.05) 


(.04) 


(.04) 


Pre-test Score 


46*** 


go* * * 


.56*** 


71*** 


54*** 


77*** 




(.04) 


(.07) 


(.03) 


(.06) 


(.03) 


(.05) 


Absent 21 


-.08** 


-.02 


-.06* 


.05 


-.05- 


-.03 




(.02) 


(.03) 


(.02) 


(.03) 


(.02) 


(.02) 


Class Size 


-.01 


-.03 


-.03 


.01 


-.05** 


-.00 




(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


(.01) 


Free/Red. Lunch 


-.11* 


.03 


-.07 


-.04 


-.08 


-.03 




(.04) 


(.05) 


(.04) 


(.05) 


(.04) 


(.04) 


LEP 


.04 


-.05- 


.04 


-.01 


.02 


.01 




(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


Special Education 


_ 09* * * 


- 16** 


.01 


-.14* 


_ 12*** 


-.00 




(.02) 


(.05) 


(.03) 


(.05) 


(.02) 


(.04) 


Mobility 


-.12* 


-.07 


-.07 


-.06 


-.13- 


-.01 




(.05) 


(.04) 


(.05) 


(.04) 


(.05) 


(.04) 


School Grade 


.09** 


-.01 


.05 


.02 


08** 


.01 




(.03) 


(.03) 


(.03) 


(.03) 


(.03) 


(.02) 


School Size 


.00 


-.02 


-.01 


-.03 


-.04 


.01 




(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


(.02) 


Minority 


-.14*** 


-.07 


- 13** 


-.12* 


_ j_2** 


-.04 




(.04) 


(.04) 


(.04) 


(.04) 


(.04) 


(.03) 


Hillsborough 


.19* 


.15- 


21** 


.25** 


-.09 


.06 




(.07) 


(.07) 


(.07) 


(.07) 


(.07) 


(.06) 


Orange 


-.12 


-.08 


-.09 


.12 


_ 23* * * 


-.10 




(.07) 


(.07) 


(.07) 


(.07) 


(.06) 


(.06) 


Broward 


-.12 


.01 


47*** 


.13 


-.00 


-.08 




(.07) 


(.07) 


(.07) 


(.07) 


(.07) 


(.06) 


Lee 


.06 


.03 


-.02 


-.07 


-.26** 


.01 




(.09) 


(.08) 


(.09) 


(.09) 


(.09) 


(.07) 


Palm Beach 


-.03 


.01 


.15 


.11 


-.22* 


-.05 




(.08) 


(.08) 


(.08) 


(.08) 


(.08) 


(.07) 


Polk 


.03 


.01 


.15 


-.05 


_ 29* * * 


-.05 




(.07) 


(.08) 


(.08) 


(.08) 


(.07) 


(.07) 


Pinellas 


-.23** 


-.21* 


-.15- 


.17 


-.19* 


-.02 




(.07) 


(.08) 


(.07) 


(.08) 


(.07) 


(.06) 


R2 


.78 


.91 


.76 


.92 


.77 


.93 



- p<.10, * p < .05, ** p < .01, *** p < .001 after Bonferroni correction 



